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Abstract — The theory of belief functions is one of the most 
important tools in information fusion and uncertainty reasoning. 
Dempster’s rule of combination and its related modified versions 
are used to combine independent pieces of evidence. However, 
until now there is still no solid evaluation criteria and methods 
for these combination rules. In this paper, we look on the evidence 
combination as a procedure of estimation and then we propose 
a set of criteria to evaluate the sensitivity and divergence of 
different combination rules by using for reference the mean 
square error (MSE), the bias and the variance. Numerical 
examples and simulations are used to illustrate our proposed 
evaluation criteria. Related analyses are also provided. 

Keywords — belief functions; evidence combination; evaluation; 
sensitivity; divergence. 

I. Introduction 

The theory of belief functions, also called Dempster-Shafer 
evidence Theory(DST) [1], is one of the most important 
theories and methods in information fusion and uncertainty 
reasoning. It can distinguish ‘unknown’ and ‘imprecision’ and 
propose a way to fuse or combine different pieces of evidence 
by using the commutative and associative Dempster’s rule of 
combination. 

Dempster’s rule of combination can bring counter-intuitive 
combination results in some cases [2], [3], so there have 
emerged several improved and modified alternative evidence 
combination rules, where counter-intuitive behaviors are im- 
puted to the combination rule itself, especially the way to 
deal with the conflicting mass assignments. The representative 
works include Yager’s rule [4], Florea’s robust combination 
rule (RCR) [5], disjunctive rule [6], Dubois and Prade’s rule 
[7], proportional conflict redistribution rule (PCR) [8], and 
mean rule [9], etc. 

As aforementioned, several combination rules are available 
including Dempster’s rule and its alternatives. Then, how 
to evaluate them? This is crucial for the practical use of 
the combination rules. The qualitative criterion is that the 
combination results should be intuitive and rational [10]. Up to 
now, there is still no solid performance evaluation approaches 
for combination rules, especially for establishing quantitative 
criteria. In this paper, we propose to interprept the evidence 
combination as a procedure of estimation [11]; therefore, a 
combination rule is regarded as an estimator. So, we define 
some statistical criteria on sensitivity and divergence for the 
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different combination rules by using for reference the idea of 
Mean Square Error (MSE) and its decomposition in estimation. 
By adding small errors to the original pieces of evidence 
(i.e., the “input” of the “estimator”), we check the mean 
square error, the variance, and the bias of the combination 
result (“output” of the estimator) caused by adding some 
noise to describe the sensitivity and divergence of the given 
combination rule. Distance of evidence [12] is used in our 
work to define the variance, the bias and other related criteria. 
Simulation results are provided to illustrate our proposed 
evaluation approaches. Dempster’s rule and major available 
alternative rules are evaluated and analyzed using the new 
evaluation approaches. 



II. Basics of DST 

Dempster-Shafer evidence theory (DST) [1] has been 
developed by Shafer in 1976 based on previous works of 
Dempster. In evidence theory, the elements in frame of discern- 
ment (FOD) 0 are mutually exclusive and exhaustive. Define 
in : 2 e — » [0, 1] as a basic belief assignment (BBA, also called 
mass function) satisfying: 



m(A) = 1, m(0) = 0 (1) 

Ae 2 e 

If m(A) > 0, A is called a focal element. In DST, two reliable 
independent bodies of evidence (BOEs) toi(-) and m 2 (-) are 
combined using Dempster’s rule of combination as follows. 
VA e 2 e : 

f O,A = 0 

m(A) = { T.A i nB j =A m r(A i )m2(B j ) A /flt (2) 
[ ~ • A r W 

where 

K = m i(^i W b j) (3) 

represents the total conflicting or contradictory mass assign- 
ments. Obviously, from Eq. (2), it can be verified that Demp- 
ster’s rule is both commutative and associative. For Dempster’s 
rule of combination, the conflicting mass assignments are 
discarded through a classical normalization step. 
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As firstly pointed out by Zadeh [2], Dempster’s rule 
has been criticized for its counter-intuitive behaviors 1 . DST’s 
validity has also been argued [3], There have emerged several 
alternatives of evidence combination rules aiming to suppress 
the counter-intuitive behaviors of classical Dempster’s rule. 
See [8] for details. 

To measure the dissimilarity between different BBAs, the 
distance of evidence can be used. Jousselme’s distance [13] is 
one of the most commonly used distance of evidence, which 
is defined as 



d J (m 1 ,m 2 ) = y - • (mi - m 2 ) T Jac (mi - m 2 ) (4) 

where the element J t] = Jac(A, ,Bj) of Jaccard’s weighting 
matrix Jac is defined as 

= (5 » 

There are also other types of distance of evidence [12], [14]. 
We choose to use Jousselme’s distance of evidence in this 
paper, because it has been proved to be a strict distance metric 
[15]. 

III. Some major alternative combination rules 

In this section, some major combination rules in evidence 
theory other than Dempster’s rule are briefly introduced. For 

all Ae2 e 

1) Yager’s rule [4]: 
m(0) = 0 

in Y a.ger(A) = Y m 1 {B i )m 2 {Cj) 

B i r\C J =A^m ( 6 ) 

m(0) = mi(0)m 2 (0) + Y mi {B t )m 2 (Cj) 
snc=0 

In Yager’s rule, the conflict mass assignments are assigned to 
the total set of the FOD 0. 

2 ) Disjunctive rule [ 6 ]: 
m(0) = 0 

m Dis (A) = Y mi(Bi)m 2 (Cj) (7) 

BiUCj=A 

This rule reflects the disjunctive consensus. 

3) Dubois & Prade's rule (D&P rule) [7]: 
m(0) = 0 

m DP {A) = Y mi(Bi)m 2 (Cj) 

BiCCj=A^® ( 8 ) 

+ mi(Bi)m 2 (Cj) 

B i C\C j =<b,B i UCj=A 

This rule admits that the two sources are reliable when they 
are not in conflict, but only one of them is right when a conflict 
occurs. 



According to the viewpoint of proponents of Dempster’s rule, the counter- 
intuitive behavior is imputed to the sensors, the data or the BOEs obtained 
from different sources, but not to Dempster’s rule itself. 



4) Robust Combination Rule (RCR, or Florea’s rule) [5]: 

m.RCR^A) = a(K)m Dis (A) + j 3(K)m Co nj(A ) (9) 

where mm s is the BBA obtained using the disjunctive rule, 
tnconj is the BBA obtained using the conjunctive rule, and 
a(K), P(K) are the weights, which should satisfy 

a{K) + (1 - K)P(K) = 1 (10) 

where K is the conflict coefficient defined in Eq. (3). Robust 
combination rule can be considered as a weighted summation 
of the BBAs obtained using the disjunctive rule and the 
conjunctive rule, respectively. 

5 ) PCR5 [8 ]: Proportional Conflict Redistribution rule 
5 (PCR5) redistributes the partial conflicting mass to the 
elements involved, in the partial conflict, considering the 
canonical form of the partial conflict. PCR5 is the most 
mathematically exact redistribution of conflicting mass to non- 
empty sets following the logic of the conjunctive rule. 

mpcRb{$) = 0 

and MX e 2 e \ {0} 

'ntpcRb(A) = ^ mi(Xi)m 2 (X 2 )+ 

X 1 ,X 2 62 e 

x 1 nx 2 =x 

mi(X) 2 ?n 2 (X 2 ) m 2 (X) 2 mi(X 2 ) 

“ mi(X) +m 2 (X 2 ) m 2 (X) +mi(X 2 ) 

X2(=2 

x 2 nx=(fi 

In fact there exists another rule PCR6 that coincides with 
PCR5 when combining two sources, but differs from PCR5 
when combining more than two sources altogether and PCR6 is 
considered more efficient than PCR5 because it is compatible 
with classical frequentist probability estimate [16]. 

6) Mean rule [9]: 

1 % Ti 

m mean (A ) = - > rm(A) (12) 

n z — '*=1 

By using this rule, we can find the average of the BBAs to be 
combined. 

For the purpose of the practical use of different com- 
bination rules, the evaluation criteria are required. In the 
next section, the available evaluation criteria or properties of 
evidence combination rules are beriefly introduced. 

IV. Properties of combination rules as 

QUALITATIVE CRITERIA 

1) Commutativity [17]: The combination of two BBAs mi 
and m 2 using some rule R does not depend on the order of 
the two BBA, i.e., 

R(mi,m 2 ) = R(m 2 ,mi) (13) 

All the combination rules aforementioned in Section III are 
commutative. 
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2) Associativity [17]: The combination result of multiple 
BBAs does not depend on the order of the BBAs to be 
combined. For example, when there are 3 BBAs, 

m 2 ), TO 3 ) = f?(TOi, m 3 )) (14) 

Dempster’s rule and disjunctive rule are associative. The other 
rules introduced in Section III are not associative. The property 
of associativity is important to facilitate the implementation of 
the distributed information fusion system. But it should be 
noted that it is not necessarily efficient in term of quality of 
fusion result. Non-associative rules are able to provide better 
performances in general than associative rules [16]. 

3) Neutral impact of the vacuous belief [17]: The combi- 
nation rule preserves the neutral impact of the vacuous BBA, 
i.e., when to 2 is m( 0 ) = 1 , 

f?(mi,m 2 )=mi (15) 

All the rules aforementioned in Section III but the mean rule, 
satisfy this property. 

These criteria are qualitative and they correspond to good 
(interesting) properties that a rule could satisfy. It should be 
noted that these “expected good” properties do not warrant 
that a real efficient fusion rule must absolutely satisfy them. 
Therefore, these properties are not enough to the evaluations of 
combination rules. In this paper, we propose some quantitative 
evaluation criteria for combination rules. 

V. Statistical sensitivity and divergence of 

COMBINATION RULES 



combination results after adding small noise to the original 
BBA to reflect the sensitivity and divergence of a combination 
rule. If under a given small noise, a combination rule bring 
out smaller variance and smaller bias, then such a rule is less 
divergent and less sensitive, based on which, the sensitivity 
and divergence of combination rules can be evaluated. The 
definitions of MSE, variance and bias for combination rules, 
and the evaluation procedure are as follows. 




Estimate 




New Evidence 



Fig. 1. Evidence combination and Estimation. 



Step 1: Randomly generate a BBA to. Add random noise to 
to for N times, respectively. In each time, the noise is e,; (small 
values), where i = 1 The noise sequence is denoted 

by e = [ei, e 2 , ..., ejv]. Here each Ci is a small real number 
(negative or positive) close to zero. Then, we can obtain a 
sequence of noised BBAs as 



Here, we develop a group of criteria for combination rules 
in terms of sensitivity and divergence. The idea of Mean 
Square Error (MSE) and its decomposition are used as a basic 
framework for such a development. 

A. Mean Square Error and its decomposition 

For an estimate x of the scalar estimand x, the MSE is 
defined as 

MSE(i) = E[(x — x) 2 ] (16) 

MSE can be decomposed as 

MSE(i) = E[{x - E(x)) 2 } + E[(E(x) - *) 2 ] 

= Var(i) + (Bias(x, x)) 2 

The MSE is equal to the sum of the variance and the squared 
bias of the estimator or of the estimations. The variance can 
represent the divergence of the estimation results. The bias can 
represent the sensitivity of the estimator. 

B. Criteria for statistical sensitivity and divergence 



m' = [toi, to 2 , ..., toat] (18) 

It should be noted that all the noised BBAs are normalized. 

Step 2: Generate original combination results sequence 
with a combination rule R 



m, = [to), to 2 



c , »ic/ c , 



m; 



N 1 



= [ R{m , to), R(m, to), ..., R(m, to)] 
The length of m c is N. 



(19) 



Step 3: Generate combination results sequence by combin- 
ing BBAs with noise and the original BBAs using the rule of 
R 



m cn = [m) n , to 2 „, ..., m^ n ] 

= [R(mi,m), f?(m 2 , to), ..., R(itin , rn )] 



( 20 ) 



Step 4: Calculate the MSE of m c 



as 



N 



MSE BB A(m c „) = [dj(m’,TO’J]' 



i= 1 



= ^E i= i [dj{R{m,m), R(m,mi))Y 



(21) 



If we consider the procedure of evidence combination with 
a given rule as an estimator (as illustrated in Fig. 1), then we 
can consider the combination results as the estimations. 



where dj is Jousselme’s distance defined in Eq. (3). MSEbba 
represents the error between the original combination results 
and the results obtained using BBAs with noise. 



So, we can use for reference the MSE and its decompo- 
sitions to measure the error, the variance, and the bias of the 
combination results based on the given combination rule. Here 
we attempt to design some criteria related to the sensitivity and 
divergence of combination rules. We use the change of the 



We can also calculate the relative MSE by removing the 
effect of the noise amplitude as follows 



MSEbba ( m cn) 



MSEbba (m cn ) 

nP 



( 22 ) 
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Step 5: Calculate the variance of m c „ as 



Var BBA (m c „) = 1^] [dj{m l cn ,m cn )\ 



= jE \dj{R{m,mi),± J2 



where m cn = A Y.j=i m in = Var B BA 

represents the fluctuations of the combination results obtained 
using BBAs with noise. 

Then, calculate the relative variance by removing the effect 
of the noise amplitude as follows 



Var BBA (m cn ) = 



Var BBA (m 

cn ) 

Var(e) 



Relative variance in fact represents the degree of amplifica- 
tion or reduction of the variances between and after combina- 
tion. In the evaluation procedure, for all the combination rule, 
the variance of the noise are the same (using the same noise 
sequence for different rules). So, high relative variance values 
also represent high divergence among all the combination 
results using a given combination rule when adding noise. 
Small relative variance values are preferred, which represent 
the high cohesion of a given combination rule. 

In this work, we propose a statistical evaluation approach 
for evidence combination rules based on Monte-Carlo simula- 
tion. To implement the statistical evaluation of a combination 
rule according to the method introduced here, two problems 
should be resolved at first. One is the way of adding noise and 
the other is the way of random generation of BBA. 



Relative variance in fact represents the degree of am- 
plification or reduction of the variances between and after 
combination. 



Step 6: Calculate the bias of m cn as 




N 



= dj(R(m., m), A J2 m j )) 

j= i 

Bias BB A represents the difference between the expectation of 
the combination results obtained using BBAs with noise and 
the original combination results. 

Then, calculate the relative bias by removing the effect of 
the noise amplitude as follows 

tj. / / \ Bias BB A(m C n) . 

Bias BBA (m c „) = pH (26) 

Regenerate randomly a new original BBA m for M times. 
In each time, re-do Step 1 to Step 6. Based on the M groups 
of results, calculate the averaged MSE BBA , the averaged 
Vai'gBA’ an d the averaged Bias BBA . These three indices are 
called the statistical MSE, the statistical variance, and the 
statistical bias of the combination rule R. We jointly use 
these indices (quantitative criteria) to describe the statistical 
sensitivity and divergence of a given combination rule R. 

Relative MSE is a comprehensive index. Larger relative 
MSE intuitively means larger sensitivity. However, relative 
MSE is insufficient to evaluate a combination rule. So we 
should further use its decomposition (including the relative 
variance and the relative bias) for a deeper analysis. 

High relative bias values represent high sensitivity, it rep- 
resents high degree of departure from the origin. It can reflect 
a given combination rule’s capability of sensitive response to 
the changes in input evidences. It represents the “agility” of a 
combination rule. Moderate relative bias values are preferred, 
which means the balance or trade-off between the robustness 
and the sensitivity. 



C. Method I for adding noise 

Method I for adding noise is designed to evaluate the effect 
of the slight value change of the mass of the existing focal 
element. Suppose that m is a BBA defined on FOD 0. First, 
we find the primary focal element (the focal element having the 
highest mass assignment) 2 , i.e., the focal element A,; satisfying 

i = arg max ?n(Aj) (27) 

j,Ajce 

Second, add the noise e to the mass assignment of the 
primary focal element. 

m'(Ai) = m{A i ) • (1 + e) (28) 

Then, for the mass assignments of other focal elements in 
original BBA, 

mfAj) = m(Aj) - . e . m(A;), Vj ± i (29) 

1 TYly/±i J 

m! is the generated BBA with noise. It is easy to verify that 

E B ce m ' (S) = 1 (30) 

It can be seen that the change of mass assignment for the 
primary focal element is the most significant when compared 
with those of other focal elements. The change of the mass 
assignment for primary focal element is redistributed to all the 
other focal elements according to the ratio among their corre- 
sponding mass assignments. BBAs are generated according to 
Algorithm 1 below [12], 

For method I for adding noise, some restrictions should be 
adopted for the values of original BBA and the noise added 
to make sure that the noised BBA m' satisfies the defintion of 
BBA. The restriction are as shown in Eq. (31). 

0 < (1 + e) • max(m(A)) < 1, VA e 2 e (31) 

A 

2 For example, when 0 = {61,^2} and m({0i}) = 0.8, m({02}) = 
0.1, m(0) = 0.1, the primary focal element is When © = {0\ , 62 \ 

and m({9 1}) = O.45,m({02}) = 0.45, m(0) = 0.1, the primary focal 
elements are { 0 \ } and {(>2 }■ 
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Algorithm 1. Random generation of BBA 

Input: 0: Frame of discernment; 

Nmax : Maximum number of focal elements 
Output: Output: to: BBA 
Generate V{&), which is the power set of 0; 
Generate a random permutation of V(&) —> 77.(0); 
Generate an integer between 1 and N max — > l; 
FOReach First k elements of 77(0) do 
Generate a value within [0, 1] —> nrii, i = 1, 1; 
END 

Normalize the vector m = [mi, to;] —> m'; 
m(Ai) = to'; 

where m is the original BBA. 



D. Method II for adding noise 

Method II for adding noise is designed to evaluate the effect 
of creating new focal elements. Suppose that m is a BBA with 
a special structure defined on FOD 0. The focal elements are 
some singletons {0;} and the total set 0. First, find out a pair 
of singletons {0j} and { 0 3 }. 

Second, create a new focal element {9i,9j} with the mass 
value of e, i.e., m'({0j, 0j}) = e. 

Then, the mass values for focal elements {9i} and {9j} 
are regenerated as 






= m({9i}) - e • 
= "i({0j}) - e ' 



™(W})+™({ 9 j}) 

™({0j 1 ) 

m({8 i })+m({6 j }) 



(32) 



Obviously, one has J^bco m '(B) = 1- 

The BBAs with special structure (with only some 
singletons and the total set focal elements) are generated 
according to Algorithm 2 below: 



Algorithm 2. Random generation of BBA 



Input: 0: Frame of discernment; 
n: Cardinality of 0; 

Nmax'- Maximum number of focal elements 
Output: to: BBA 

Generate V{&), which is the power set of 0; 

Generate a random permutation of V{&) —> 72.(0); 

FOR i = 1 : Nmax - 1 

Generate an integers j between 1 and n; 

Generate a focal element F t : {9j}', 

END 

Generate a focal element Fm : 0. 

1 y max 

FOR i = 1 : Nmax 

Generate a value within [0, 1] — >• mp, 

END 

Normalize the vector to = [mi, .... rriM, nax } — > to'; 
m{F.j) = to'; 

For method II for adding noise, some restrictions should 
be adopted for the values of original BBA and the noise added 
to make sure that the noised BBA to' satisfies the defintion of 
BBA. According to Eq. (32), the restriction can be obtained as 



shown in Eq. (33). For all the available singleton focal element 
{9i} in original BBA, 



0 < m({9i}) ■ (1 



e 



) 



where to is the original BBA. 



< 1,VA G 2 e 
(33) 



E. A simple illustrative example 

Here an illustrative example of single cycle calculation of 
the evaluation indices is provided by using Method I for adding 
noise. By referring to this illustrative example, evaluations by 
using Method II are easy to implement. 

A BBA to defined on the FOD 0 = {9i, 02, ^3} is 
?n({0i}) = 0.6, m({9 2 }) = 0.3, m({9i, 9 2l 9 3 }) = 0.1. 

Suppose that the noise sequence is 
e = [-0.1, -0.05, -0.02, 0.02, 0.05, 0.1]. 

It can be seen that the restrictions in Eq. (31) are not 
violated. 

According to the Step 1, we generate the sequence six 
noised BBA m' = [mi, TO2, m 3 , mu, TO5, me] as follows: 

mi({6h}) = o.5,mi({0 2 }) = 0.375, mi ({6»i, 0 2 , 6 * 3 }) = 0.125; 
m 2 ({6»i}) = 0.55, m 2 ({0 2 }) = 0.3375, m 2 ({6»i, 6» 2 , 0 3 }) = 0.1125; 
m 3 ({6h}) = 0.58, m 3 ({0 2 }) = 0.315, m 3 ({6h , 62 , 0 3 }) = 0.105; 
m 4 ({e»i}) = 0.62, m 4 ({0 2 }) = 0.285, m 4 ({0i, 02, 03>) = 0.095; 
m 5 ({0i}) = 0.65, m 5 ({0 2 }) = 0.2625, m 5 ({0i, 0 2 , 0 3 }) = 0.0875; 
m 6 ({0i}) = 0.7, m 6 ({0 2 }) = 0.225, m 6 ({0i,0 2 ,0 3 }) = 0.075. 

Here we use Dempster’s rule of combination. Then, ac- 
cording to the Step 2, the original combination sequence 

m c = [ml, m 2 c , ..., to®] is as 

Vi = 1, ..., 6. 

m*({0i}) = 0.75, m'({0 2 }) = 0.2344, 62 0 3 }) = 0.0156. 

Then according to the Step 3, the sequence of combination 
results by combining BBAs with noise and the original BBAs 

m cn = [ml n ,m? cn , ...,m 6 c J is as 

mL({ 8 i}) = 0.6627, mL({0 2 }) = 0.3162, 

ml n ({e 1,02,0s}) = 0.0211; 

m 2 cn ({0 1 }) = 0.7076, rr&({0 2 }) = 0-2741, 

m 2 cn({9i, 02, 0 3 }) =0.0183; 

mL({9i}) = 0.7334, toL({0 2 }) = 0.2500, 

mL({0 1,02,03}) =0.0167; 

mL({ 6 1 }) = 0.7662, toL({0 2 }) = 0.2192, 

mtn({0 1,02,0s}) = 0.0146; 

m 5 cn ({0 1 }) = 0.7895, mL({02» = 0.1973, 

mL({0i, 02, 0 3 }) = 0.0132; 

m 6 cn ({9 1 }) = 0.8257, toL({0 2 }) = 0.1634, 

m® n ({0i, 02, 0 3 }) =0.0109. 

For the noise sequence, 

|| e|| 2 = 0.0258 
Var(e) = 0.0043 

According to the Step 4, the value of MSE is 
MSE BB A(m cn ) = 0.00270 
MSE BBA (m cn ) = 0.00270/0.0258 = 0.104752 
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According to the Step 5, the value of variance is 
Var B BA(m cn ) = 0.002697 
Var BBA (m c „) = 0.002697/0.0043 = 0.6272 
In the final, according to the Step 5, the value of bias is 
Bias BB A(m c „) = 0.002406 
Bias BBA (m c „) = 0.002406/\/0.0258 = 0.014978 



TABLE III. Comparisons in terms of Bias 



Combination Rules 


Bias BBA 


Rank 


Dempster’s rule 


0.81132*10“' 


1 


Yager’s rule 


0.59228*10“ 7 


2 


Disjunctive rule 


0.41949*10“ 7 


4 


D&P rule 


0.49075*10“ 7 


3 


RCR 


0.39592*10“ 7 


5 


PCR5 


0.38066*10“ 7 


6 


Mean rule 


0 


7 



The above is the illustration of one-cycle procedure. One 
can use other combination rules to do these steps. Randomly 
generate BBAs and repeat all the steps, then we can obtain the 
final statistical evaluation results. 



VI. Simulations 

A. Simulation I: using Method I for adding noise 

In our simulations, the cardinality of the FOD is 3. In 
random generation of BBAs, the number of focal elements 
has been set to 5. The length of the noise sequence is 50 (the 
noise value starts from -0. 1 , with an increasing step of 0.004, 
up to 0.1. Of course, the zero value for noise is not considered 
because it corresponds to noiseless case.). In each simulation 
cycle, seven combination rules including Dempster’s rules 
and other alternatives aforementioned in Section III are used, 
respectively. We have repeated the Monte Carlo simulation 
with 100 runs. In random generation of original BBAs, the 
restrictions in Eq. (31) are not violated. The statistical results 
are listed in Tables I-III. The ranks of the relative MSE, relative 
variance and relative bias are obtained based on the descending 
order. 



It should be noted that when using RCR in our simulation, 
the weights are generate as follows. 



f a(K) — 1K 
l P{K) = l-A'+ A ' 2 



(34) 



of relative bias, and it is always a rule with smaller divergence 
according to the criterion of the relative variance. Yager’s rule 
is always more sensitive to the mass change and is always not 
so divergent. PCR5 rule is not so sensitive to the mass change 
according to the criterion of Bias (rank 6), and it is not so 
divergent according to the criterion of the relative variance. 
The Robust combination rule (RCR), Dubois & Prade’s rule 
(D&P rule) are always moderate to the mass change in terms 
of sensitivity and in terms of divergence. So, PCR5 and RCR 
are more moderate rules; thus, they are relatively good choices 
for practical use. 

B. Simulation II: using Method II for adding noise 

In our simulations, the cardinality of the FOD is 3. In 
generation of BBAs, the total set 0 is used as a focal element 
and the number of singleton focal elements has been set to 
2. The length of the noise sequence is 50 (the noise value 
starts at 0.002 with an increasing step of 0.002, up to 0.1.) 
In each simulation cycle, seven combination rules including 
Dempster’s rules and other alternatives aforementioned in 
Section III are used, respectively. We have repeated the Monte 
Carlo simulation with 100 runs. In random generation of 
original BBAs, the restrictions in Eq. (33) are not violated. The 
statistical results are listed in Tables IV- VI. The ranks of the 
relative MSE, relative variance and relative bias are obtained 
based on the descending order. 

The derivation of weights of RCR has been done in the 
same manner as for the Simulation I. 



TABLET Comparisons in terms of MSE 



Combination Rules 


mse^ ra 


Rank 


Dempster’s rule 


0.0010758 


1 


Yager’s rule 


0.0005743 


5 


Disjunctive rule 


0.0004298 


7 


D&P rule 


0.0006247 


4 


RCR 


0.0007152 


3 


PCR5 


0.0010505 


2 


Mean rule 


0.0005711 


6 



TABLE II. Comparisons in terms of Variance 


Combination Rules 


V ar RRA 


Rank 


Dempster’s rule 


1.7260 


1 


Yager’s rule 


0.9568 


6 


Disjunctive rule 


0.7469 


7 


D&P rule 


1.0226 


4 


RCR 


1.2788 


3 


PCR5 


1.6801 


2 


Mean rule 


1 


5 



As we can see in Tables I - III, Dempster’s rule are the 
most sensitive to the mass change according to the criterion of 
the relative bias, and it also has highest degree of divergence 
according to the criterion of relative variance. Mean rule is the 
most insensitive to the mass change according to the criteria 



TABLE IV. Comparisons in terms of MSE 



Combination Rules 


mse^ ra 


Rank 


Dempster’s rule 


0.0050 


3 


Yager’s rule 


0.0038 


5 


Disjunctive rule 


0.0033 


6 


D&P rule 


0.0014 


7 


RCR 


0.0078 


1 


PCR5 


0.0043 


4 


Mean rule 


0.0056 


2 



TABLE V. Comparisons in terms of Variance 


Combination Rules 


V ar RRA 


Rank 


Dempster’s rule 


0.9218 


3 


Yager’s rule 


0.7019 


5 


Disjunctive rule 


0.5434 


6 


D&P rule 


0.2714 


7 


RCR 


1.3528 


1 


PCR5 


0.8014 


4 


Mean rule 


1 


2 



As we can see in Tables IV - VI, RCR is the most 
sensitive to the change of focal elements according to the 
criterion of the relative bias, and it also has highest degree 
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TABLE VI. Comparisons in terms of Bias 



Combination Rules 


BiasR RA 


Rank 


Dempster’s rule 


0.0016 


3 


Yager’s rule 


0.0012 


5 


Disjunctive rule 


0.0011 


6 


D&P rule 


0.0004 


7 


RCR 


0.0025 


1 


PCR5 


0.0013 


4 


Mean rule 


0.0018 


2 



of divergence according to the criterion of relative variance. 
Dubois & Prade’s rule (D&P rule) is the most insensitive 
rule according to the criteria of relative bias, and it is always 
a rule with smaller divergence according to the criterion of 
the relative variance. Yager’s rule is always insensitive and is 
always not so divergent. Mean rule is sensitive to the change of 
focal element according to the criterion of Bias (rank 2), and it 
is divergent according to the criterion of the relative variance. 
Dempster’s rule is not so sensitive to the change of focal 
element. The PCR5 and Yager’s rules are always moderate 
to the change of focal elements in terms of sensitivity and in 
terms of divergence. 

According to simulations results, we see that the different 
methods of adding noises impact differently the results of the 
comparative evaluations. However, we have shown that no 
matter the method adopted (by keeping the original core of the 
BBA, or modifying it slightly), PCR5 provides quite robust 
results for combining two BBA’s and thus offers practical 
interests from this standpoint. 

VII. Conclusion 

In this paper we have proposed a group of statistical criteria 
for evaluating the sensitivity of different combination rules 
with respect to the noise perturbations. The design is based 
on the classical measures of performance like MSE, variance, 
and bias encountered in the estimation theory. We don’t rank 
the rules according to their a priori “good expected” properties. 
Moderate relative bias values are preferred, which means the 
balance or trade-off between the robustness and the sensitivity. 
Small relative variance values are preferred, which represent 
the high cohesion of a given combination rule. Seven widely 
used evidence combination rules were evaluated using the new 
proposed evaluation criteria. PCR5 is a moderate rule which 
is good for the practical use for combining two BBAs. For 
combining more than two BBAs, we expect that PCR6 will 
be a good choice, but we need to make more investigations in 
future to evaluate precisely its performances. 

In this work, we have added some noises to BBAs mainly 
by modifying the mass assignments of the primary focal 
element and by creating new focal elements. In our future 
work, we will try to use other methods to add noise to BBAs, 
e.g., eliminating some of original focal elements. In our Monte- 
Carlo simulations, there is no pre-settings of mass assignments 
for the BBAs. In this paper, in each cycle we only generate one 
BBA, based on which, we generate a sequence of BBAs by 
adding small noise. The BBAs to be combined are the original 
BBAs and the BBAs with small noise. In our future work, 
we will try to generate two BBA sequences and add noise 
to them, respectively, where we can use some special BBAs 
in the evaluation procedure, e.g., BBAs to be combined are 
high conflicting. Then we can do more specific performance 



evaluations on the combination rules. In this paper, we did 
only focus on the property of sensitivity and divergence. The 
evaluation criteria of other aspects of evidence combination are 
also required for evaluating and designing new combination 
rules, which will be investigated in future research works and 
forthcoming publications. 

Acknowledgment 

This work was supported by the Grant for State Key Pro- 
gram for Basic Research of China (973) No. 2013CB329405, 
National Natural Science Foundation (No. 61104214, No. 
61203222), Specialized Research Fund for the Doctoral 
Program of Higher Education (20120201120036), Shaanxi 
Province Science and Technology Research and Development 
Program (2013KJXX-46), and Fundamental Research Funds 
for the Central Universities. 

References 

[1] G. Shafer. A Mathematical Theory of Evidence, Princeton, NJ: Princeton 
University Press, 1976. 

[2] L. A. Zadeh. “A simple view of the Dempster-Shafer theory of evidence 
and its implication for the rule of combination,” AI magazine, 1986, vol. 
7, no. 2, p. 85-90. 

[3] J. Dezert, P. Wang, A. Tchamova. “On the validity of Dempster- 
Shafer theory,” In: Proceedings of the 15th International Conference on 
Information Fusion, Singapore, July, 2012, p. 655-660. 

[4] R. R. Yager. “On the Dempster-Shafer framework and new combination 
rules,” Information Science, 1987, vol. 41, no. 2, p. 93-137. 

[5] M. C. Florea, A.-L. Jousselme, E. Bosse, D. Grenier. “Robust combina- 
tion rules for evidence theory,” Information Fusion, 2009, vol. 10, no. 2, 
p. 183-197. 

[6] P. Smets. “Belief functions: the disjunctive rule of combination and the 
generalized Bayesian theorem,” International Journal of Approximate 
reasoning, 1993, vol. 9, no. 1, p. 1-35. 

[7] D. Dubois, H. Prade. “Representation and combination of uncertainty 
with belief functions and possibility measures,” Computational Intelli- 
gence, 1988, vol. 4, no. 3, p. 244-264. 

[8] F. Smarandache, J. Dezert. Applications and Advances of DSmT for 
Information Fusion (Vol3). Rehoboth: American Research Press, 2009. 

[9] C. K. Murphy. “Combining belief functions when evidence conflicts,” 
Decision Support Systems, 2000, vol. 29, no. 1, p. 1-9. 

[10] R. Haenni. “Are alternatives to Dempster’s rule of combination real 
alternative?: Comments on ‘About the belief function combination and 
the conflict management problem’ ,” Information Fusion, 2002, vol. 3, 
no. 4, p. 237-239. 

[11] Y. Bar-Shalom, X. Rong Li, T. Kirubarajan. Estimation with Applica- 
tions to Tracking and Navigation. USA, New York: John Wiley & Sons, 
2001. 

[12] A.-L. Jousselme, P. Maupin. “Distances in evidence theory: Comprehen- 
sive survey and generalizations,” International Journal of Approximate 
Reasoning, 2012, vol. 53, no. 2, p. 118-145. 

[13] A.-L. Jousselme, D. Grenier, E. Bosse. “A new distance between two 
bodies of evidence,” Information Fusion, 2001, vol. 2, no. 2, p. 91-101. 

[14] D. Q. Han, Y. Deng, C. Z. Han, Y. Yang. “Some notes on betting 
commitment distance in evidence theory,” Science China - Information 
Sciences, 2012, vol. 55, no. 3, p. 558-565. 

[15] M. Bouchard, A.-L. Jousselme, P.-E. Dore. “A proof for the positive 
definiteness of the Jaccard index matrix,” International Journal of Ap- 
proximate Reasoning, 2013, vol. 54, no. 5, p. 615-626. 

[16] F. Smarandache, J. Dezert. “On the consistency of PCR6 with the 
averaging rule and its application to probability estimation,” In Proc. 
of Fusion 2013 Int. Conference on Information Fusion, Istanbul, Turkey, 
July 9-12, 2013, p. 1127-1134. 

[17] F. Smarandache, J. Dezert. Applications and Advances of DSmT for 
Information Fusion ( Vol2 ). Rehoboth: American Research Press, 2006. 



229 




